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This paper reports partial outcomes from a three-year project that provided professional 
learning opportunities in mathematics for middle school teachers in rural schools in 
Tasmania. The educational environment for the study was one of significant system 
transition. Student change is reported here and was measured with survey instruments 
reflecting the basic elements of numeracy considered essential to students’ development of 
critical quantitative thinking and preparation for study of further mathematics. Student 
improvement was significant across grades in the project. 



Hiebert and Grouws (2007) take the phrase “opportunity to learn” as the foundation for 
their suggestions for effective teaching likely to influence student learning. While 
acknowledging the influence of the curriculum and subject matter, they go on to explore 
specific connections related to teaching for skill efficiency and for conceptual 
understanding. In terms of teaching for skill efficiency they identify the following teacher- 
centered features: “teaching that facilitates skill efficiency is rapidly paced, includes 
teacher modelling with many teacher-directed product-type questions, and displays a 
smooth transition from demonstration to substantial amounts of error free practice” (p. 
382). In relation to teaching for conceptual understanding, which they define as the process 
of creating “mental connections among mathematical facts, procedures and ideas” (p. 382), 
Hiebert and Grouws suggest two key features. The first is that teachers and students attend 
explicitly to concepts, in a public way. The second is that students struggle to make sense 
of important mathematics that is within reach containing key ideas that are comprehensible 
but not yet fully formed. These key features for skill efficiency and conceptual 
understanding, together with implications for teacher professional learning, are reflected in 
the teaching and learning goals of the project reported in this paper. 

Sowder (2007), in her extensive review of the mathematical education and 
development of teachers advocated goals of ongoing professional learning reflecting 
Shulman’s (1987) types of teacher knowledge. Specifically, her summary of elements of 
successful professional development includes the importance of the following components: 

determining the purpose of a ... program, the role of teachers in deciding on foci, . . . the need to 
have support from other constituencies ... to undertake changes in instruction, ... collaborative 
problem solving, . . . continuity over time, . . . modelling the type of instruction expected, and . . . 
assessment that provides teachers with feedback ... (p. 171) 

The work of Hawley and Valli (1999) reflected that of Sowder (2007), but with added 
emphases on teachers having initial and continuing input to the professional learning 
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program and the opportunity to work collaboratively on problem solving. These elements 
further informed the project reported here. 

This paper reports on one aspect of a three-year professional learning program for 
teachers, supported with funding and in-kind contributions from the state government and 
Catholic school systems. Implementation of the program “Mathematics in an Australian 
Reform-Based Learning Environment” (MARBLE), in 2005, coincided with the 
introduction of the Essential Learnings Framework (Department of Education, Tasmania 
(DoET), 2002, 2003). In 2006 however, amid controversy over the implementation of the 
Essential Learnings, a new curriculum, “The Tasmanian Curriculum”, was announced to 
“make it [curriculum] easier to understand, and more manageable for teachers and 
principals” (DoET, 2007, para 1). 

Using a measurement instrument developed based on the seven types of teacher 
knowledge of Shulman (1987) as formalised by Watson (2001), teachers were surveyed at 
the beginning and end of the program, and students in their classes were surveyed in each 
of the three years. The focus of this paper is on the change in student performance as a 
result of the teachers’ professional learning program and therefore specific features of the 
teacher profiles and interview and students’ beliefs and attitudes are not discussed here. 

Methodology 

Sample 

The teachers in this study were working in nine schools that were chosen by the two 
participating education systems. The schools were in two rural clusters in different 
geographical regions of the state, divided five and four. Eight were government schools 
and one was a Catholic school. Initially there were 42 teachers in the project teaching 
Grades 5 to 8. In the second year of the project there was a total of 47 teacher participants, 
of whom only 23 had participated in the previous year. In the final year of the project, 
there were 54 teacher participants, of whom 20 were new to the project. On completion 
only 19 teachers had participated throughout the 3 years. The numbers of students in the 
project surveyed each year are shown in Table 1. Students completed a two-part survey, 
Part B of which is discussed in this paper. Students were surveyed either once, twice or 
three times depending on the grade they were in at the beginning of the project. 

Table 1 

Student Sample Sizes for Part B of the Student Survey (numbers in parenthesis indicate 
students surveyed two or three times) 



PART B 


2005 


2006 


2007 


Grade 4 


23 (5 a ) (15 b ) 


17 (13 a ) 


20 


Grade 5 


186 (39 a ) (1 16 b ) 


180 (142 a ) (14 b ) 


212 (13 a ) 


Grade 6 


222 (70 a ) (99 b ) 


187 (59 a ) (115 b ) 


227 (145 a ) (14 b ) 


Grade 7 


181 (56 a ) (53 b ) 


196 (9 1 a ) (90 b ) 


203 (46 a ) (11 6 b ) 


Grade 8 


130 


143 (70 a ) (53 b ) 


162 (52 a ) (90 b ) 


Grade 9 


3 




110 (45 a ) (53 b ) 



a Students surveyed twice. b Students surveyed three times. 
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Instruments 



Part B of the survey was written to reflect five foundation concepts of middle school 
mathematics identified in the literature: Number Sense, Proportional Reasoning, 
Measurement, Uncertainty, and Relationships. Of the 38 distinct items in the first year 
survey, there was overlap in terms of items reflecting these concepts. The items, and 
rubrics, had various sources including Watson and Callingham (2003), Callingham and 
Griffin (2000) and the Department of Education, Community and Cultural Development 
(1997). Student outcomes for one of the problems based on fractional parts of a nebulous 
whole were discussed in Watson, Beswick, and Brown (2006). The second student survey 
contained 34 distinct items in Part B, 8 items in common with the initial survey, again 
across the five foundation concepts. Student outcome levels from the initial surveys and 
the nature of the intervention with teachers as a part of the professional learning program 
in 2006 influenced the nature of items in the second survey. Again, items were scored 
using rubrics as for the initial survey. The third student survey contained 40 distinct items 
designed to measure mathematical performance, covering the range of mathematical 
concepts covered earlier. All items were taken from one or other of the preceding two 
surveys with seven being common to both. They were scored using the same rubrics as in 
the preceding two years. An example of one of the common items relating to proportional 
reasoning and the rubric used to assess it are presented in Table 2. 



Table 2 

Example of a Proportional Reasoning Item and the Rubric for Scoring 



A table can seat six people: two (2) on each side and one (1) on each end. 



Q Q 

© 

© © 



© 



When tables are put together, more people can be seated (as shown here). 



© 



© © © © 



© 

© © © © 



Write a rule for the number of people ( p ) that can be seated at a certain number of tables (t). 



Code 


Global Category 


Example 


3 


Correct rule 


4 (p) per (t) then add 2 on the ends 
Four for each table then add 2 
If there were 6 tables, times it by 4 then add 2 
For 10 table, 10x4 + 2 


2 


Partial explanation of 
the rule 


When you join tables together you can only have 2 at the ends all the 
time. 


1 


Inappropriate 
explanation of the rule 


People can be seated 6 to a table. 


0 


No explanation 





Professional Learning 

The MARBLE professional learning project was devised in the context of Tasmania’s 
Essential Learnings (DoET, 2003) to assist teachers in providing middle school students 
with the mathematical foundation necessary for the quantitative literacy needs of today’s 
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society (Steen, 2001), as well as for the further study of mathematics and contribution to 
innovation in Australia (Committee for the Review of Teaching and Teacher Education, 
2003). The underlying model is shown in Figure 1. 

The features of successful 
professional development identified by 
Sowder (2007) were incorporated in the 
project: the education systems and 
schools were very supportive, teachers 
were consulted about their needs on 
several occasions over the three years, 
there was the continuity over three 
years, the leaders attempted to model 
the teaching strategies advocated, and 
opportunities were provided for 
collaborative problem solving. All 
schools had several teachers involved in 
the project and there was the 
expectation that they would work 




Figure 1. MARBLE framework for mathematical 
learning. 



collaboratively in their schools as well as when they were at project learning sessions. In 
addition, Key Curriculum Press provided its middle school statistical software, TinkerPlots 
(Konold & Miller, 2005), to all schools in the project. 

Professional learning was delivered in two ways. Whole of cluster sessions were 
combined with case studies, where each school was assigned a researcher to be involved in 
a project of its own choice (e.g., Beswick, 2009; Brown, Rothwell, & Taylor, 2007). 
Ongoing feedback was sought from participating teachers: at the ends of the sessions, in 
meetings with school-based coordinators, through surveys of teachers leaving the project 
and by way of interviews with 19 teachers at the conclusion of the project. 

By the end of the project, a total of 24 whole of cluster professional learning sessions 
had been provided, 3 in the first year, 1 1 in the second, and 1 0 in the final year of the 
project. Whole of cluster sessions were largely replicated in each cluster, however different 
needs expressed by the teachers in each cluster resulted in some modifications to content 
and format. 



Data Analysis 

A performance measure of numeracy ability was obtained from Part B of the student 
survey. As stated previously, there were 38 distinct items in Part B of the 2005 survey, 34 
in 2006 and 40 in 2007. Seven items were common to the three surveys. Using the same 
measurement techniques as reported in Watson, Beswick, Brown, and Callingham (2007), 
data from the mathematics tasks were analysed using the Rasch Partial Credit Model 
(Masters, 1982). The seven link items common to all three surveys provided an anchor set 
that established the difficulties of the items at each test administration relevant to each 
other (Griffin & Callingham, 2006). Estimates of person ability were identified for each 
student in 2005, 2006 and 2007, anchored to the same set of link item difficulties. In so 
doing, genuine comparisons could be made and these ability measures were used as a basis 
for subsequent analyses. The performance of students in each grade was summarised for 
each year of the project and these measures provided a comparison of performance by 
grade. Furthermore, summary information from students who completed all three tests 
provided a measure of growth across time. 
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Results and Discussion 



Student change in numeracy ability was analysed in two ways. First, the mean ability 
by grade was obtained for each year of the study for the full cohort of students. This 
analysis provided within grade measures, and was useful in identifying how the 
performance of the student participants differed over time. The information is shown in 
Figure 2. The first aspect is that students in the later years of the project are of considerably 
higher measured ability. Of those students who participated in 2005, only just over half 
also participated in 2006, and in 2007, 58% of students had participated in either one or 
both of the previous years. This suggests that although the overall number of students 
remained high, each year there was a large inflow of students into the project and so the 
change in ability at different grades could in part be due to the changed cohort. 




■ Ab2005 ■ — Ab2006 - A - Ab2007 



Figure 2. Student performance by grade across time. 

The graph in Figure 2 shows a “learning trajectory” based on measured ability in 
different grades. In 2005, the classic transition effect (Anderman & Midgley, 1997) is 
clearly seen between Grades 6 and 7. This effect appears to have disappeared in the 2006 
and 2007 data. Because of the change in the cohort care must be taken when interpreting 
this finding, however, it does suggest that the project helped to reduce this effect. 

The results of a second set of analyses based on matched data only across all 3 years of 
the study are shown in Table 3. For each grade cohort the mean ability score increased 
from 2005 to 2006 and from 2006 to 2007. Large effect sizes and very strong statistical 
significance values are evident in all cohorts with the exception of the Grade 6 2006 to 
Grade 7 2007 cohort. This suggests that very positive change in student performance in the 
mathematical thinking section of the surveys occurred from 2005 to 2007. 

Figure 3 shows the growth over time of these students as they moved from grade to 
grade, and includes 259 students across Grades 5 to 9. The legend shows the start grade in 
2005. Again there are a number of features worthy of note. The slopes of the lines are very 
similar, indicating that, in general, the rate of growth for these students was similar 
regardless of the grade in which they started the project. The growth for the students who 
started the project in Grade 7 in 2005 to 2006 (Grade 8) is very similar to the growth of 
students from lower grades. There is, however, almost no growth from 2006 (Grade 8) to 
2007 (Grade 9). Grade 9 was not a target grade for the study, and the teachers of this grade 
did not participate in the professional learning program. It is possible that the lack of 
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growth among this group is associated with a changed teaching as the students moved into 
the higher grade, but without additional data this idea has to remain as a conjecture. 



Table 3 

Performance by Grade across Time - Part B items 



Outcome Measure 
(Grade, Y ear) 


Initial Ability (logits) 


Final Ability (logits) 


t 


P 

value 


Effect 

Size 


mean 


n 


SD 


mean 


n 


SD 




G5 2005 - G6 2006 


-0.80 


116 


0.94 


0.43 


90 


0.68 


10.55 


0.000 


1.48 


G6 2006 - G7 2007 


0.43 


90 


0.68 


0.52 


53 


1.38 


0.53 


0.594 


0.09 


G6 2005 - G7 2006 


-0.28 


90 


0.70 


0.45 


53 


1.05 


4.98 


0.000 


0.86 


G7 2006 - G8 2007 


0.45 


53 


1.05 


1.02 


90 


0.84 


3.57 


0.000 


0.61 


G7 2005 - G8 2006 


-0.28 


53 


0.87 


0.43 


90 


0.68 


5.43 


0.000 


0.93 


G8 2006 - G9 2007 


0.43 


90 


0.68 


0.74 


116 


0.92 


2.68 


0.008 


0.37 




Figure 3. Student ability measures, growth over time (matched students only) 
shown by start grade in 2005 (error bars omitted for clarity). 

Among the professional learning experiences of the researchers and teachers (as 
reported formally in teacher interviews on completion of the project, and informally 
throughout the life of the project), a few sessions stand out as examples that may have been 
instrumental in influencing the outcomes for students. These include sessions on 
measurement, proportional reasoning, mathematical inquiry, pattern and algebra, and 
fractions, some of which have been reported elsewhere (Watson, 2008; Watson & Wright, 
2008; Brown, Watson, & Wright [in press]; Watson, Skalicky, Fitzallen, & Wright, 2009). 
Teachers particularly appreciated that the professional learning sessions incorporated 
practical ideas, example test items and sequencing of activities. Teachers also noted that 
the researchers’ emphasis on student understanding was particularly beneficial with many 
commenting that following the professional learning intervention they subsequently 
checked for student understanding prior to moving onto other or more complex areas of 
mathematical study. Watson et al. (2006) describe the change in the teacher’s reported 
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level of knowledge of their students and their own ability to intervene when issues of 
misunderstanding occur. 

Another important aspect of the professional learning program that may have affected 
student learning was the process of involving teachers in the analysis of student survey 
responses and the consideration of the implications for classroom teaching. Feedback from 
teachers was that they had never previously undertaken this type of activity and that they 
developed meaningful intuitions by so doing. This process may have been particularly 
influential in the final year of the project where significantly improved outcomes were 
reported, as teachers and researchers had the results of the previous two years to work with 
and professional learning was focused specifically on areas of weakness as indicated in the 
2005 and 2006 survey results. 

In terms of the students themselves, teachers commented that by the end of the project 
their students were more able to explain their mathematical thinking, their confidence in 
their own ability had increased, and they were asking more questions than ever before. Of 
equal, if not greater, importance is that teachers reported that their students now enjoyed 
mathematics more than they did previously. 

Conclusion 

The MARBLE project was underpinned by the principles accepted in the literature as 
appropriate foundations for teacher professional learning with the potential to effect 
improved student numeracy learning outcomes (e.g., Hiebert & Grouws, 2007; Sowder, 
2007), and improved outcomes did occur for students over the three years of the project. A 
comparison of these outcomes with those reported in Watson et al. (2007), which focused 
on performance during the first two years of the project, shows distinct improvement in the 
students’ mathematical thinking. Although disappointing outcomes were seen across 
cohorts of students in the same grades, with the exception of Grade 7, in the first two years, 
by the end of the project statistically significant improvement was seen in most cohorts 
(the exception being the Grade 6 students in 2006 moving to Grade 7 in 2007, in which the 
improvement was in a positive direction but not at a significant level). The continued and 
vastly improved growth over time of matched students is also encouraging and lends 
support to the authors’ belief in the positive influence of the teacher professional learning 
on student learning. The project represents a step towards the improving the evaluation of 
professional learning programs to include measures of the outcomes that matter most: 
students’ mathematical understandings. 
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